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Assessing Computer Use ...on Instructional Transformation 



The Levels of Computer Use assessm.ent (LCU) was developed to classify 
teachers* use of computers so that vaiiations of use could be studied. It was derived 
fix)m the model of Instructional Transfonnation (Rieber & Welliver, 1989; Welliver, 
1990). The development of the LCU was stimulated by the identification of the 
discrepancy between advocacy for the use of computers in education and their actual 
use by teachers. 

It is widely believed that computer technology can help improve the educational 
system (see The National Task Force on Educational Technology, 1986; United 
States Office of Technology Assessment [OTA], 1988; Sheingold and Hadley, 1990; 
Shanker, 1990). At the same time, the availabihty of computers in schools has 
increased. In the period fi*om 1983 to 1987, the average pupU-to-computer ratio in 
public elementary schools improved from 112.4 to 36.8 pupils per computer (OTA). 

Despite the positive trends in computer availabihty, relatively few teachers have 
integrated computers into teaching. A recent national survey has shown that even in 
schools where computers were available at more than double the national average, 
only about one teacher per school had integrated the computer into the classroom 
(Becker, 1989; Sheingold & Hadley, 1990). Apparently, sheer availability of 
computers did not result in their increased integration into teaching. 

Indeed, there is a strong sentiment of disappointment and disillusionment 
among teachers regarding educational technology (Becker, 1987; Bjerkhe & Hollis, 
1991;Dronka, 1985; Ely & Plomp, 1988; O'NeiU, 1990; Roblyer, 1988). Hie poor 
opinion of educational computing by teachers does not agree with the views of it^ 
supportei's and patrons in government, academe, and teachers' unions. This 
discrepancy in perceptions of edu'^ational computing deserves study because the 
result of it is that computers in education are underutilized. As a result, whatever 
hopes there are for educational computing to affect education may never be realized. 
For the research on teachers' computer use, a fundamental question is, "What is 
meant by use?' This question has been addressed in the model of Instructional 
Transformation suggested by Rieber and Welliver (1989). 



Instructional Transformation 



The model of Instructional Transfonnation describes the process of the adoption 
of innovation. This is an area of human behavior that has been studied generally— 
types of adoption behavior based on length of time have been described (Rogers, 
1983; Rogers & Shoemaker, 1971) and specifically— a framework for analyzing the 
adoption of innovations among educators has been described (Hall, Loucks, 
Rutherford, & Newlove, 1975). Tlie model of Instructional Transformation focuses 
on the adoption of a specific educational innovation— tlie computer— and describes a 
teacher^s progression thiwgh five stages of involvement witli computers (Rieber and 
WeUiver, 1989; Welliver, 1990). First, a teacher becomes familiar with computers 
(familiarization )] then, the teacher uses computers in teaching (utilization). A higher 
level of use is observed when the computers have become critical to the teaching 
(integration). An analogy to this scenario would be of one who upon learning to drive 
(familiarization), uses the car to improve one's general transpoiiation in order to 
fuliill a variety of pvuposes (utilization). One may then schedule the use of the car so 
that the vehicle is critical to the function of fulfilluig the purposes (integmtion). In 
education, this stage includes the teacher*s awareness of the change of his or her role. 
With this new awareness of the restructuring of teaching activities, the teacher 
pursues an expansion and fme tuning of the computer-teacher-student relationship 
(reorientation). Tne final stage (evolution) is more of a suggestion than a condition, 
and that is to continue practicing and learning about how to develop the 
improvement of instruction through the systematic implementation of computer 
technology. 

The definition of use for the Levels of Computer Use assessment was derived 
from tlie model of Instructional Transformation. Use was defined as the integrated 
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employment of computers in teaching. Two levels of use were designated— Utilization 
and Integration. At the Utilization level, a teacher shares and delegates teaching 
duties to the computer; however, the absence of the computer would not prevent the 
implementation of iiistruction. At the Integration level, teaching duties would also be 
shared by and delegated to the computer; however, the absence of computers would 
prevent the implementation of instruction. The distinction between tlie Utilization 
and Integration levels lies in the expendability of the computer technology. 
Expendability describes the relationship of computer technology to a teacher's 
plaimed instruction— whether or not instruction would be able to continue in the 
hypothetical event of, say the sudden absence of computer technology. It is this 
dimension of expendability that was identijfied as the cut-off for membership in either 
category. According to the model of Instructional Transfonnation, the Integration 
stage is further characterized by tlie dimension of a teacher^s emergent self- 
awareness of a role cl ange in teaching from teacher-centered to learner-centered. In 
a departure from the model, the assessment does not encompass the dimension of 
awareness of role change. 



Development of the Assessment 



The LCU is the result of several attempts at varying formats. In the original 
conception of the assessment, items were written for each of the five stages of the 
model of Instructional Transformation. This had been field tested and revised several 
times and was finally supplanted by the current format. The earlier forms of the 
assessment adhered to the progressive nature of the model and attempted to captui^e 
all of its dimensions, however, it was difficult to classify responses. This is because 
there are several dimensions that run throughout the model in a progressive 
overlappiag manner. In the current form, one dimension is assessed. The most 
appropriate technique for this sort of assessment is the paired comparisons 
technique. TTiis technique requires that a discrete definition be operationalized to 
determine the cut off for the levels of use. Additionally, the levels need to be mutually 
exclusive and exhaustive. Nimnally (1959) considers tins method "the most exact 
psychophysical tool..." useful for precise information concerning judgments or 
preference. 

In the format of the LCU, each item from one level is paired alternately with 
each item from the other level (See Table 1). For example, if there were four items, 
two from the first level (la, lb) and two firom the second level (2a, 2b), the paired 
comparisons would be (la/2a), (la/2b), (lb/2a), and (lb/2b). Tlie respondent is forced 
to endorse one item per pair. The paired comparisons technique allows only one 
combination of responses to indicate membership at either level. There are four pairs 
of items in this version. Two items indicate the Utilization level and two the 
Integration level. If, for example, a value of 1 were assigned to responses indicating 
the Utilization level, consistent responses for 4 items would score 4 points. A value of 
2 for responses indicating the Integration level, would result in consistent response 
scoring of 8. Acceptable scores would be 4 or 8. Scores of 6 or 7 indicate an 
^ inconsistency. The four items were selected fix)m a field testing with a sample of 50 
elementary school teachers because those items had the highest response 
consistency. In that testing, there were 15 items. Items which generated response 
inconsistency or which introduced an additional dimension were eliminated. 

In a subsequent field testing, twenty-three elementary school teachers 
responded; there were 2 inconsistent responses. The estimated reliability of the LCU 
for the second field trial using the CoejOTicient of Reproducibility ( CR) was .74. One of 
the items was rewoi^ed. The LCU was then administered to 170 elementary school 
teachers. The CR was estimated at .96. A CR of .90 is the criterion for demonstrating 
that items form an ordered scale of allowable response patterns (Crocker & Algina, 
1986). Hie teachers were also asked to respond to a control item about their self- 
reported computer use; the responses were matched with those of the LCU and 
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provided additional data for estimating criterion-related validity. The consistency of 
classification of the measure was estimated computing Cohen's kappa {Kappa = .72). 
The results are important for two reasons: 1) the estimated reliability is high, and 2) 
the demonstration of an ordered scale suggests that the model of Instructional 
Transfonnation identifies at least two progressive levels. 



Table 1 

The Levels of Computer Use assessment 



1. 


*a. 


In my instruction, the use of the microcomputer is supplemental. 




•b. 


The microcomputer is critical to the functioning of my instruction. 


2. 


*a. 


The use of the microcomputer is not essential in my instruction. 




•b. 


t or my teaching, the use of the microcomputer is indispensable. 


3. 


•a. 


The microcomputer is critical to the fimctioning of my instruction. 




*b. 


The use of the microcomputer is not essential in my instruction. 


4. 


•a. 


For my teacliing, the use of the microcomputer is indispensable. 




*b. 


In my instruction, the use of the microcomputer is supplemental. 


Note: 


• 


indicates an item of the integration level, 
indicates an item of the utilization level. 



The LCU has been used in several studies. It has been used to establish a 
dependent variable— level of computer use— in a study of elementary school teachers 
(MarcinMewicz, 1991, in press) and in a longitudinal study of preservice 
undergraduate teachers (Marcinkiewicz & Grabowski, 1991). It has been adapted by 
the Grosse Pointe (Michigan) Public School System for use in its Staff Computer 
Skills Siuvey (J. Corbett, personal communication, February 21, 1992). It is being 
used to estabUsh a dependent variable— levels of computer use— relative to the 
subjective norms of rural teachers. It should be noted that prior to this writing, the 
assessment has been referred to as tiie LU assessment. As of this discussion, the 
assessment is referred to as the LCU— Levels of Computer Use— assessment. This 
modification has been made to reflect the purpose of the assessment more accurately. 
This modiJacation also vmderscores the ongoing study of the assessment and of the 
model of Instructional Transformation. 



Discussion 



There is a fundamental notion in computer implementation that is oflen 
overlooked— not all intended computer users do use them. As with most innovations, 
people vary in their adoption of computer use. By focusing on a specific profession 
such as teaching, the study of how its members— teachers— differ in their use of 
computers is more manageable. They differ before tiiey begin to use computers and 
they differ once tiiey have begun to use them. Before use, tiiey differ in the time it 
takes to adopt the idea of using the technology. After teachers adopt the notion of 
using computers, they differ in the nature of their computer use. '[Oie importance of 
tiie model of Instructional Transformation is tiiat it addresses the fundamental 
variance in the adoption of computer use. It describes the progression of a teacher 
from tiie onset of the adoption of computers. In application, this theoretical model 
will help educators in several ways: 1) to identify the range of expejcted behaviors 
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app)licable to a t^acher^s computer use— what can we expect a teacher to become?; 2) 
to identify tlie level of a teacher's computer use; 3) to identify whether a teacher is 
progressing in computer use; 4) to identify how the nature of a teacher's instruction 
changes due to his or her computer use; 5) to identify how the nature of teacher's 
professional self-perception changes due to computer use, 

When educators can identify these aspects of teachers, tlie professional growth 
and development of teachers relative to computer use can be understood. With this 
vmderstanding there is a basis for recommending staff development, remediation, or 
differential staf&ng. The LCU assessment was developed so that educators could 
implement the model of Instructional Transformation. Some applications of the 
assessment include assessing a schcors teachers to determine the current computei* 
utiUzation/underutilization levels before a purchase order for computers is issued, so 
that the likelihood of future use can be estimated. Also, as a school is designing its 
restructuring plans, the LCU can be used in needs assessment to identify 
discrepancies in the levels of computer use. A school may also study the assessment 
and the model of Instructional Transformation to learn what the extent of 
possibihties of computer use in education are. In doing so, a school can be helped in 
clarifying its goals in the application of computer technology. 

The LCU accomplishes the classification of teachers into two levels of computer 
use. Vvhen used for tMs pxirpose, the assessment addresses point #2 mentioned 
above. The current form of the LCU is markedly different from earlier versions. The 
way in which the current and past forms differ needs to be pointed out to highlight 
the strength of the current version as well as to identify directions for future 
research. Most importantly, the past and current versions differ in that the former 
comprised several dimensions— as does the model— attempting to measure all 
dimensions. This proved to be confoimding since the dimensions share 
characteristics. It was decided that it was not practical or perhaps possible to capture 
all the dimensions simultaneously. In the LCU assessment, there is one dimension— 
the expendability of computers —that is measured. This is the central dimension of 
the model. Measures of tiie other dimensions need to be developed; such as for the 
degree of awareness of the changing of one*s role a^: a teacher. 

The successful development of the LCU estabUshes it as a part of the 
assessment arm of the model of Instructional Transformation. The model is useful in 
understanding a teacher*s theoretical progression in the adoption of computers. The 
assessment oflfers a means for applying the theoretical model. 



References 



Becker, H. J. (1987). The impact of computer use in childi^n's leaiming: W^hat 
research has shown and what it has not . (Report No. 18) Baltimore, MD: The 
Johns Hopkins University, Center for Research on Elementary and Middle 
Schools. 

Becker, H. J. ("*989). [U. S. Administration of the 1989 lEA computers in education 
survey, Johns Hopkins University Center for Social Organization of Schools]. 
Unpubhshed raw data. 

BjerkUe, D. & Hollis, R. W. (1991, May 20). The revolution that fizzled. Time , pp. 
48-49. 

Crocker, L. & Algina, J. (1986). Introduction to classical and modem test theory . 

New York: Holt, Rinehart, and Winston. 
Dronka, P. (1985). Computer integration into instruction is stuck: Experts blame 

unclear optimal uses and three implementation problems. ASCD Update . 27(5). 

1-8. 

Ely, D. P., & Plomp, T. (1988). The promises of educational technology. InB. 
Branyan-Broadbent & R. K. Wood (Eds.), Educational media and technolo^ 
yearbook 1988 . 5-18. Englewood, CO: Libraries Unlimited. 



Marcinkiewicz & Welliver 




Assessing Computer Use . . .on Instructional Transformation 



HaU, G. E., Loucks, S. P., Rutherford, W. L., & Newlove, B. W. (1975). Levels of 

use of the iimovation: A framework for analyzing innovation adoption. The 

Journal of Teacher Education . 26(1), 52-56. 
Marcinkiewicz, H. R. (in press). Computers and teachers: Factors influencing 

computer use in tlie classroom. Journal of Research on Computing in Education . 
Marcinkiewicz, H. R. (1991). The relationships of selected personolopical variables 

to the use of available microcomputers bv elementary school teachers . Doctoral 

dissertation, The Pennsylvania State University. 
Marcinkiewicz, H. R. & Grabowski, B. L. (1991). The relationsliips of 

per^onological variables to computer use by elementary school teachers: Report 

of Phase One— Baseline Data. Proceedings of Selected Research Paper 

Presentations 1992, AECT. 
National Task Force on Educational Technology. (1986). Transforming American 

education: Reducing the risk to the nation, a report to the Secretary of Education, 

United States Department of Education. T.H.E. Journal 14(1), 58-67. 
Nimnally, J. C. (1959). Testa and measurements: Assessment and prediction . New 

York: McGraw-Hill. 

O'Neill, J. (1990). Computer ^Revolution' on hold: Despite scattered progress, few 
signs of broad-based changes seen. ASCD Update . 32(9), 1-5. 

Rieber, L. P. & Welliver, P. W. (1989) Infusing educational teclmology into 
mainstream educational computing. International Journal of Instructional 
Media , 16(1), 21-32. 

Roblyer, M. D. (1988). The effectiveness of microcomputers in education: A review of 
the research for 1980-1987. T. H. E. Journa l. 16(2), 85-«9. 

Rogers, E. M. (1983). Diffusion of innovations . (3rded.). New York: The Free Press. 

Rogers, E. M. & Shoemaker, F. F. (1971). Commimication of innovations: A cross- 
cultural approach . New York: llie Free Press. 

Shaiiker, A. (1990, September 23 ). Equipping teachers for pix)ductivity: From 
chalkboards to computers. The New York Times . (Section E), p. 7. 

Sheingold, K. & Hadley, M. (1990). Accomplished teachers: intepyating computers 
into classroom practice . New York: Bank Street College, Center for Technology 
in Education. 

U. S. Congress, Office of Technology Assessment, Power on! New tools for teaching 
and learning , OTA-SET-379, Washington, DC: U.S. Government Printing Office, 
September, 1988. 

Welliver, P. (1990). Instructional transformation: A model for chan^^e . (The 
Pennsylvania State University Regional Computer Resource Center report 
to the state centers and member teachers). University Park, PA: ITEC Center 
at Penn State . 



5 

Marcinkiewicz & Welliver 



ERIC 



